DQ003 - Data Falsification Prevention and Investigation

DQ003 - Prevention and Investigation of Possible Falsification of Data
Review Committee: Data Effective Date:10/01/2019
Attachments: Revised Date: 10/01/2019
Forms: OP001 Data Collection Integrity Pledge Reviewed Date: 6/13/2024

Introduction:

Maintaining high level of data quality is a prime objective for all Model Systems but occasionally, a situation may arise that requires investigation of possible fabrication or falsification of data.

Purpose:

To provide strategies to prevent misconduct in data collection and to provide a course of action to be taken when falsification of data is suspected.

Scope:

All current TBIMS centers.

Responsibilities:

All current TBIMS Project Directors and Data Managers.

Strategies to Prevent Misconduct in Data Collection

  1. Staff Recruitment and Hiring (as your Human Resources Department allows):

    1. Look for gaps in previous employment that might indicate jobs that are not included, and flesh out the history until it’s “continuous.”

    2. If supervisors in one or more previous positions are not offered as references, inquire whether it would be OK to contact them as well (as your Human Resource Department allows). If the applicant describes problematic behaviors on the part of that supervisor that would make them a “bad reference,” offer reassurance that that will be taken into consideration, but that you still need to confirm employment.

    3. Listen to your gut: If a candidate seems “too smooth” or sets off inauthentic radar, don’t take that as evidence, but do take it as a reason to probe about issues of honest communication and trust with prior employers.

  2. Training and Reiteration (Data Center and Model System Centers):

    1. “Provide training on the importance of adhering to and maintaining the integrity of data collection protocols” (Murphy 2016).

    2. Provide a workplace environment that “encourages honesty, discourages falsification, enhances morale, and values data quality (American Statistical Assoc, 2003)

    3. Once trained, require data collectors to sign the Data Integrity Pledge to reinforce the importance of high quality, reliable data.

  3. Staff Supervision:

    1. Hold routine discussions on balancing the desire for productivity with the desire for rigor and accuracy. Don’t create a climate where staff feel that they need to meet certain targets “at all costs.” Ask data collectors what challenges they are having in meeting targets in an open-ended and collaborative way that emphasizes problem solving. Give positive feedback to staff who come forward with obstacles to enrollment or data collection, or who identify their own mistakes. Routinely give examples of your own challenges and seek input from others, to role model openness in problem solving.

    2. Consider having more than one staff member perform some role in each activity, so that there is always more than one person who knows the mechanics of that function, and so that periodic spot checking of each other’s work is facilitated.

    3. Avoid housing staff alone, particularly in areas remote from casual foot traffic “supervision”, until their work patterns are clear.

    4. Institute random and unannounced data and file audits, as a matter of routine, so they are not perceived as indicators of suspicion but simply as “good practice” for data completeness and accuracy.

    5. Provide positive feedback from monitoring process with the team.

Procedural Steps for Investigating Possible Falsification of Data:

  1. Assessment: Determine if data falsification happened. To determine if data falsification happened, and to define the extent of falsification:

    1. Document timeline of suspected behavior

    2. Compile list of records that were potentially affected

    3. Crosscheck resources: date of interview, date of returned mail-out questionnaire, date of data entry, data collector phone logs, any other paper trail

    4. Look for excessive data entry within 1 week of data submission and/or records with large gap between collection date and data entry

  2. Notify the NDSC and NIDILRR TBIMS Program Manager and Project Officer within 10 business days of identifying data falsification and provide routine updates of findings and actions taken. At minimum, provide an update within 10 business days from first notifying the NDSC and NIDILRR TBIMS Program Manager and Project Officer. Provide a summary of extent of data manipulation (timeframe, number of forms potentially falsified, number of forms deleted/updated, any personnel action taken, future plan to avoid similar incidence)

  3. Rectify data that is potentially corrupted (delete or update). The NDSC will work closely with PI and flag datasets to remove identified records.

Detailed Instructions for Review of Data Sources

Form 1:

  1. List cases that may have been affected for dates indicated.

  2. Review consent forms

    1. Present

    2. Complete

    3. Witness signatures are present when required

    4. Signatures appear to be valid

  3. Look for agreement between the following dates (dates happen in correct order e.g. consent date prior to payment date, consent date prior to data entry date;

    1. Informed consent date

    2. Payment date (for those centers that reimburse for consent)

    3. Data entry date

  4. Compare signatures on consent forms and payment forms (for those centers that reimburse for consent)

  5. If consent form is incomplete or missing, verify eligibility and re-consent eligible participants.

  6. If unable to re-consent, delete Form 1 data.

  7. Review data for accuracy.

Form 2:

  1. Establish cutoff date for review

  1. Consider when pressure to perform may have begun and/or when Tracking Report/Missing Report numbers improved

  2. Include cases with a large gap (greater than 3 weeks) between interview date & entry date

  3. Include cases where excessive data entry occurred within 1 week of data submission

  1. Look for agreement between the following dates:

  1. Date of Interview

  2. Phone logs

  3. Contact forms/logs

  4. Payment/reimbursement logs

  5. Any other paper trail (such as telephone company call records)

  1. Call participants/significant others who have a Form 2 in question and recent contact (within 3 months) to verify interview and/or payment for interview. Double-check more stable variables, such as address, employment questions, and FIM physical items to compare with the most recent Form 2 data.

  1. If memory of interview – Keep Form 2

  2. If no memory of interview consider deleting Form 2

  3. If unable to follow-up with participant/significant other, consider deleting Form 2

Training requirements:

None

Compliance:

All current TBIMS centers are responsible for adhering to this policy and its procedures. If a center’s institution has policies in place that supersede or conflict with this SOP, the center’s policy will take precedent.

References:

Murphy, Joe, et al. Interviewer Falsification: Current and best practices for prevention, detection, and mitigation. Statistical Journal of IAOS. 32 (2016) 313-326

History:

Date Action
10/01/2019 New policy developed

Review Schedule:

At least every 5 years.